Image processing device, method, and storage medium for gamma correction based on illuminance

ABSTRACT

An illuminance calculation unit detects an illuminance of an image when the image is captured. Based on the detected illuminance, a gamma correction amount calculation unit corrects at least one of a minimum output luminance value and a maximum output luminance value in a gamma characteristic for correcting a gradation characteristic of a luminance of the image. A gamma setting unit corrects the gradation characteristic of the luminance of the image with the gamma characteristic corrected based on the illuminance.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an image processing device, an imageprocessing method, and a storage medium.

Description of the Related Art

Monitoring cameras are required to capture images both day and night,meaning that images need to be captured even under low illuminancecondition. Generally, images captured under low illuminance conditionare adjusted to have a gain increased by automatic gain control (AGC).

Japanese Patent Application Laid-Open No. 2005-236588 discusses atechnique for converting a minimum conversion luminance value and amaximum conversion luminance value into a luminance range of an outputimage, with an intermediate conversion luminance value set to be on acenter side of the dynamic range. The minimum conversion luminance valuehas a predetermined value close to 0% in a luminance histogram. Theintermediate conversion luminance value has a predetermined value at oraround 50% in the luminance histogram. The maximum conversion luminancevalue has a predetermined value close to 100% in the luminancehistogram. Japanese Patent Application Laid-Open No. 2004-363726discusses a technique for executing gradation value conversionprocessing, based on a gradation conversion characteristic selected inaccordance with the luminance value from a plurality of gradationconversion characteristics prepared. The gradation conversioncharacteristics feature nonlinear change in output-input characteristicsof gradation values.

An image obtained by the adjustment to increase the gain of the imagecaptured under low illuminance results in an image that is brighter butis likely to become a low image quality having a large amount of noise.The large amount of noise negatively affects the compression performanceof an encoder that performs compression based on H.264 standard and thelike, resulting in an increase in the data size of an image. Themonitoring cameras are required to record and store data for a longperiod of time, and thus such increase in the data size requires arecording medium with a large capacity leading to an increasedintroduction cost. However, for example, it is expected that an imagecaptured under low illuminance condition can be adjusted with reducedimage degradation while resultant increase in the data size of the imageis prevented, if the noise and the entropy of the image are reduced inaccordance with the illuminance condition at the time when the image iscaptured. The techniques discussed in Japanese Patent ApplicationLaid-Open No. 2005-236588 and Japanese Patent Application Laid-Open No.2004-363726 described above can achieve a luminance range of an outputimage wider than that of an input image, but cannot prevent the increasein the data size of the image while restraining image degradation.

SUMMARY OF THE INVENTION

According to an aspect of the present invention, an image processingdevice includes an illuminance detection unit, a gamma correction unit,and a processing unit, the illuminance detection unit detects anilluminance of an image when the image is captured, wherein based on thedetected illuminance, the gamma correction unit corrects at least one ofa minimum output luminance value and a maximum output luminance value ofa gamma characteristic for correcting a gradation characteristic of aluminance of the image, and the processing unit corrects the gradationcharacteristic of the luminance of the image with the gammacharacteristic corrected based on the illuminance.

Further aspects and features of the present invention will becomeapparent from the following description of exemplary embodiments withreference to the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example of a schematicconfiguration of an image processing device according to a firstexemplary embodiment.

FIG. 2 is a graph illustrating an example of a reference gammacharacteristic.

FIG. 3 is a diagram illustrating an example of a sufficient illuminanceimage.

FIG. 4 is a diagram illustrating an example of a low illuminance image.

FIG. 5 is a graph illustrating an example of corrected and uncorrectedgamma characteristics according to the first exemplary embodiment.

FIG. 6 is a diagram illustrating an example of the low illuminance imageafter gamma processing with the corrected gamma characteristic.

FIG. 7 is a table illustrating an example of a data size reduction withcorrection in the first exemplary embodiment.

FIG. 8 is a graph illustrating an example of a histogram of thesufficient illuminance image.

FIG. 9 is a graph illustrating an example of a histogram of the lowilluminance image.

FIG. 10 is a flowchart according to the first exemplary embodiment.

FIG. 11 is a graph illustrating an example of relationship between again and noise.

FIG. 12 is a table illustrating offset amounts corresponding to gains.

FIG. 13 is a diagram illustrating an example of an image as a result ofexcessively offsetting a low luminance area.

FIG. 14 is a block diagram illustrating an example of a schematicconfiguration of an image processing device according to a secondexemplary embodiment.

FIG. 15 is a graph illustrating effective luminance.

FIG. 16 is a graph illustrating an example of corrected and uncorrectedgamma characteristics according to the second exemplary embodiment.

FIG. 17 is a flowchart according to the second exemplary embodiment.

FIG. 18 is a table illustrating luminance range change ratioscorresponding to various gains.

DESCRIPTION OF THE EMBODIMENTS

Exemplary embodiments of the present invention are described withreference to the drawings.

A first exemplary embodiment of the present invention is describedbelow. FIG. 1 is a diagram illustrating an example of a schematicconfiguration of an image processing device according to the presentexemplary embodiment. The image processing device illustrated in FIG. 1can be installed in various cameras such as a monitoring camera and acamera installed in smartphones and the like. A lens group 1 illustratedin FIG. 1 includes a focus lens performing in-focus position adjustmentand a zoom lens performing focal length adjustment. For example, adiaphragm 2 is used for adjusting the amount of light incident on animage sensor 3 via the lens group 1. The image sensor 3 converts anoptical image formed on an imaging plane via the lens group 1 and thediaphragm 2 into an analog image signal. An analog-digital (AD)conversion unit 4 converts the analog image signal output from the imagesensor 3 into digital image data. The image data output from the ADconversion unit 4 is input to an image processing unit 6. An illuminancemeter 5 measures an illuminance on an object and the like when the imageis captured, and outputs the resultant illuminance data to the imageprocessing unit 6.

When exposure control is performed in the image processing unit 6 inaccordance with the object, a gain setting unit 61 performs gain controlon the image data from the AD conversion unit 4. Increased gain canachieve a brighter image of the object but causes more noise in theimage. A shutter speed setting unit 62 controls shutter speed so as toachieve an appropriate brightness of the image of the object, based onthe image data. A luminance information generation unit 63 generatesinformation about luminance Y of each pixel based on the image data, forexample. An exposure control unit 64 control a diaphragm value of thediaphragm 2, the gain set by the gain setting unit 61, and the shutterspeed set by the shutter speed setting unit 62, based on the informationabout the luminance Y generated by the luminance information generationunit 63, to achieve an appropriate value of the luminance of the object,for example. An illuminance calculation unit 65 is an example of anilluminance detection unit configured to detect, for example, theilluminance of the object when the image is captured. The illuminancecalculation unit 65 calculates the illuminance level of the object andthe like being captured, based on at least one of the illuminance datafrom the illuminance meter 5, the diaphragm value of the diaphragm 2,the gain set by the gain setting unit 61, the shutter speed set by theshutter speed setting unit 62, and the information about the luminance Yfrom the luminance information generation unit 63. The illuminanceinformation calculated by the illuminance calculation unit 65 is sent toa gamma correction amount calculation unit 67.

A gamma setting unit 66 corrects a gradation characteristic of theluminance of the image from the luminance information generation unit63. More specifically, the gamma setting unit 66 is an example of aprocessing unit that corrects the gradation characteristic of the image,through gamma processing. In the gamma processing, an output luminancevalue is obtained by correcting an input luminance value as theluminance of the image in accordance with what is known as a gammacharacteristic. A general example of the gamma characteristic isillustrated in FIG. 2. In FIG. 2, the horizontal axis represents theinput luminance value and the vertical axis represents the outputluminance value. The gamma characteristic affects the contrast andgradation of the image, and thus has a large influence on the imagequality. The gamma correction amount calculation unit 67 is an exampleof a correction unit, and corrects at least either one of a minimumoutput luminance value and a maximum output luminance value in thecorrection characteristic for correcting the gradation characteristic ofthe luminance of the image, in accordance with the detected illuminance.In the present exemplary embodiment, the gamma correction amountcalculation unit 67 calculates the correction amount with respect to thegamma characteristic, in accordance with the illuminance informationcalculated by the illuminance calculation unit 65, and causes the gammasetting unit 66 to set the gamma characteristic as a result ofcorrection with the correction amount. As described above, in thepresent exemplary embodiment, the gamma setting unit 66 executes thegamma processing based on the gamma characteristic corrected inaccordance with the illuminance information, to correct the gradationcharacteristic of the image. The correction amount for the gammacharacteristic and the gamma processing based on the gammacharacteristic as a result of the correction with the correction amountare described in detail below. An encoding unit 7 encodes, for example,a still image based on a standard such as Joint Photographic ExpertsGroup (JPEG) and encodes a moving image based on a standard such asMoving Picture Experts Group (MPEG) or H.264.

The image including a large amount of noise increases high-frequencycomponents in encoding, leading to degraded compression performance anda larger data size after the compression. It is helpful to reduce thenoise and the entropy of the image to prevent the data size of the imagefrom increasing. To that end, in the present exemplary embodiment, theluminance range is corrected in a manner described below, so that thenoise is reduced to prevent the data size from increasing while theimage degradation is reduced. In the present exemplary embodiment, theluminance range is corrected by the gamma characteristic being correctedas described above. In the present exemplary embodiment, the gammacharacteristic is corrected by offsetting at least either one of theminimum output luminance value and the maximum output luminance value inthe gamma characteristic serving as a reference. Thus, the offset amountof the gamma characteristic serving as the reference corresponds to thecorrection amount on the gamma characteristic described above.

The correction with respect to the reference gamma characteristicthrough the offsetting and the correction of the luminance range basedon correction of the gamma characteristic are described in detail below.In the present exemplary embodiment, the luminance range is corrected asfollows. Specifically, the gamma correction amount calculation unit 67calculates the correction amount (offset amount) for the gammacharacteristic based on the illuminance information from the illuminancecalculation unit 65, and the gamma setting unit 66 is caused to set thegamma characteristic as a result of the correction based on thecorrection amount.

FIG. 3 illustrates an example of a sufficient illuminance image 10. FIG.4 illustrates an example of a low illuminance image 12. A square portion11 in FIG. 3 and a square portion 13 in FIG. 4 each represent an area inwhich the noise level is measured. An image average luminance value Yillustrated in FIGS. 3 and 4 is an average value of the luminance Yacross the entire image. Each of the values of the noise level and theimage average luminance illustrated in FIGS. 3 and 4 is a referencevalue. Here, for example, the gain setting unit 61 performs theadjustment to increase the gain on the low illuminance image 12illustrated in FIG. 4. Accordingly, the noise level becomes much higherin the low illuminance image 12 illustrated in FIG. 4 than in thesufficient illuminance image 10 illustrated in FIG. 3.

In the present exemplary embodiment, gamma processing with a gammacharacteristic 21 as illustrated in FIG. 5 is executed on the lowilluminance image 12 exemplarily illustrated in FIG. 4. FIG. 5illustrates a gamma characteristic 20 that is the gamma characteristicillustrated in FIG. 2 and used as a reference for obtaining the gammacharacteristic 21. The gamma characteristic 21 is obtained by, withrespect to the reference gamma characteristic 20, offsetting the minimumoutput luminance value by a first offset amount YOFL, offsetting themaximum output luminance value by a second offset amount YOFH, andinterpolating an intermediate luminance value area between the minimumoutput luminance value and the maximum output luminance value. The firstoffset amount YOFL is set as such an offset amount that increases theminimum output luminance value of the reference gamma characteristic 20by a first predetermined amount. The second offset amount YOFH is set assuch an offset amount that reduces the maximum output luminance value ofthe reference gamma characteristic 20 by a second predetermined amount.For example, in the present exemplary embodiment, the first offsetamount YOFL corresponds to a luminance value of 16, and the secondoffset amount YOFH corresponds to a luminance value of 64. The gammacharacteristic 21 may be also obtained by combining the reference gammacharacteristic 20 with a characteristic represented by a straight lineconnecting the minimum output luminance value after the offsetting bythe first offset amount YOFL with the maximum output luminance valueafter the offsetting by the second offset amount YOFH.

The following Formula (1) represents a formula for the gamma processingbased on the gamma characteristic 21.YOUT=(255−YOFL−YOFH)×γ(YIN)/255+YOFL  Formula (1),where YOUT represents the output luminance value, YIN represents theinput luminance value, and γ( ) represents a function corresponding tothe reference gamma characteristic 20 illustrated in FIG. 2.

FIG. 6 illustrates an example of an image 14 obtained by executing thegamma processing with the gamma characteristic 21 illustrated in FIG. 5on the example image illustrated in FIG. 4 described above. A squareportion 15 in FIG. 6 corresponds to the area 13 in FIG. 4. It can beseen in the example illustrated in FIG. 6 that the image as a whole hasincreased luminance to appear as a brighter image, and the noise levelis reduced with the image quality such as gradation being prevented fromdegrading, compared with the image 12 exemplarily illustrated in FIG. 4.The noise level is reduced so that the data size after the encoding bythe encoding unit 7 is reduced compared with the case of the imageexemplarily illustrated in FIG. 4. FIG. 7 illustrates an example of howmuch the data size after the encoding is reduced in an image captured ina certain scene, as a result of correcting the luminance range with thegamma characteristic corrected as described above. More specifically,FIG. 7 illustrates a ratio of a data size in a case where the offsetamount for high and low luminance areas is “16 to 64” to a data size ina case where no correction is performed. The data size is 100% when theoffset amount for both high and low luminance areas is 0 (nocorrection). It can be seen in this example illustrated in FIG. 7 thatthe data size decreases with an increase in the offset amount for bothhigh and low luminance areas in the gamma characteristic (the correctionamount for the luminance range).

FIG. 8 illustrates an example of a histogram of the sufficientilluminance image as exemplarily illustrated in FIG. 3. FIG. 9illustrates an example of a histogram of the low illuminance image asexemplarily illustrated in FIG. 4. In FIGS. 8 and 9, the horizontal axisrepresents a luminance value, and the vertical axis represents afrequency of pixels. It can be seen in the histogram exemplarilyillustrated in FIG. 8 that the sufficient illuminance image hasluminance values distributed across the entire luminance range. On theother hand, it can be seen in the histogram exemplarily illustrated inFIG. 9 that the low illuminance image has luminance values distributedmore on a low luminance side than on a high luminance side, incomparison with the example illustrated in FIG. 8. Examples of such animage with the luminance values distributed more on the low luminanceside include the image captured in a low illuminance environment asdescribed above, and further include an image captured with a low gainset and an image obtained with exposure adjustment for setting lowexposure to appear as a low illuminance captured image.

As described above, the histogram exemplarily illustrated in FIG. 9represents an image with the luminance values distributed more on thelow luminance side and less distributed on the high luminance side. Forexample, the image quality of such an image is considered to be notlargely affected by correction on the luminance range more focusing onthe high luminance side than the low luminance side. Since the highluminance area and the low luminance area both involve noise, the noiselevel reduction effect can be more effectively increased with the imagequality prevented from degrading, with the correction amount set to belarger in the high luminance area than in the low luminance area.Accordingly, in the present exemplary embodiment, the correction of theluminance range is more focused on the high luminance area than on thelow luminance area, as described above with reference to FIG. 5.

FIG. 10 is a flowchart illustrating processing in the image processingunit 6. The processing in the flowchart illustrated in FIG. 10 isexecuted by the illuminance calculation unit 65, the gamma correctionamount calculation unit 67, and the gamma setting unit 66. For example,the processing illustrated in FIG. 10 may be implemented by a centralprocessing unit (CPU) executing an image processing program according tothe present exemplary embodiment.

After a camera starts capturing an image, the image processing unit 6starts the processing in the flowchart illustrated in FIG. 10, and theprocessing proceeds to step S1001. In step S1001, the illuminancecalculation unit 65 calculates the illuminance of the object and thelike in the captured image, based on at least one of the illuminancedata, the diaphragm value, the set gain, the set shutter speed, and theluminance value Y as described above. After the processing in step S1001is completed, the processing proceeds to step S1002.

In step S1002, the gamma correction amount calculation unit 67determines whether the illuminance value calculated by the illuminancecalculation unit 65 is equal to or lower than a first illuminance value(Yth1). In a case where the gamma correction amount calculation unit 67determines that the illuminance value is equal to or lower than thefirst illuminance value (Yth1) (YES in step S1002), the processingproceeds to step S1004. On the other hand, in a case where the gammacorrection amount calculation unit 67 determines that the illuminancevalue is higher than the first illuminance value (Yth1) (NO in stepS1002), the processing proceeds to step S1003. In step S1003, the gammacorrection amount calculation unit 67 causes the gamma setting unit 66to set the reference gamma characteristic 20 described above withreference to FIG. 5. After the processing in step S1003 is completed,the processing in the flowchart in FIG. 10 is terminated.

In step S1004, the gamma correction amount calculation unit 67calculates the above-described offset amount YOFL for the low luminancearea in accordance with the illuminance, and then the processingproceeds to step S1005. In step S1005, the gamma correction amountcalculation unit 67 calculates the above-described offset amount YOFHfor the high luminance area in accordance with the luminance, and thenthe processing proceeds to step S1006. In step S1006, the gammacorrection amount calculation unit 67 offsets the reference gammacharacteristic 20 with the offset amount YOFL for the low luminance areaand the offset amount YOFH for the high luminance area, and interpolatesthe intermediate luminance area, so that the gamma characteristic 21described above is calculated. The gamma correction amount calculationunit 67 causes the gamma setting unit 66 to set the gamma characteristic21. In this manner, the gamma setting unit corrects the luminance rangeof the low illuminance image with the gamma characteristic 21. After theprocessing in step S1006 is completed, the processing in the flowchartin FIG. 10 is terminated.

FIG. 11 is a diagram illustrating an example of a relationship betweenthe noise level of an image and the gain for the image. In FIG. 11, thehorizontal axis represents the gain, and the vertical axis representsthe noise level. It can be seen in FIG. 11 that as the gain increases,the noise level increases. As described above, a higher noise levelleads to a larger data size after the encoding, and thus as the gainincreases, the data size increases. FIG. 12 is a diagram illustrating arelationship between gains and the offset amounts for the high and thelow luminance areas described above. As described above with referenceto FIG. 7, a larger offset results in a more reduced data size, and theimage quality is not largely affected with the offset amount set to belarger for the high luminance area than for the low luminance area.Accordingly, as illustrated in FIG. 12, the data size can be effectivelyprevented from increasing with the image quality prevented fromdegrading, with the offset amount increased as the gain increases andset to be larger for the high luminance area than for the low luminancearea. In this manner, in the present exemplary embodiment, the offsetamount suitable for an increased gain is obtained, while taking balancewith the image quality degradation.

FIG. 13 illustrates an example of an image that has come out too white awhole as a result of, for example, increased black level in the imagecaused by excessive raise of the luminance range in the low luminancearea. As illustrated in FIG. 13, the luminance range excessively raisedin the low luminance area results in the image that is too white as awhole. Thus, the offset amount for the low luminance side is preferablyset to be equal to or smaller than a predetermined value set inaccordance with image quality and scenes. In the present exemplaryembodiment, to prevent the image from being too white, a secondilluminance value (Yth2) is set to be used in changing of values for thehigh luminance area and the low luminance area. More specifically, inthe present exemplary embodiment, the same offset amount is set for boththe high luminance area and the low luminance area as long as theilluminance value does not drop below the first illuminance value(Yth1). In a case where the illuminance value is lower (gain is higher),a larger offset amount is set for the high luminance area than that forthe low luminance area. In the present exemplary embodiment, the offsetamount for the high luminance area is set to be larger than that for thelow luminance area in a case where the illuminance value drops to orbelow the second illuminance value (Yth2). There is a correlationbetween the gain and the illuminance, in which the lower illuminanceraises the gain. Accordingly, as illustrated in FIG. 12, the offsetamount may be set in accordance with the gain. The increased gain leadsto lower image quality due to noise reduction and the like. Thus, in acase where the gain increases, the image quality is not largely affectedby an increased offset amount. FIG. 12 illustrates an example where thegain of 36 (dB) corresponds to the illuminance with the secondilluminance value (Yth2). The offset amount for the high luminance areais larger than that for the low luminance area under a lower illuminancecondition corresponding to the gain exceeding 36 (dB).

As described above, the image processing device according to the presentexemplary embodiment corrects the luminance range by executing the gammaprocessing with a gamma characteristic offset in accordance with theilluminance. In this manner, noise reduction can be achieved so that thedata size can be reduced with the image quality prevented fromdegrading.

A second exemplary embodiment of the present invention is describedbelow. FIG. 14 is a diagram illustrating a schematic configuration of animage processing device according to the present exemplary embodiment.Components in FIG. 14 that are the same as the counterparts in FIG. 1are denoted with the same reference numerals, and the descriptionthereof is omitted. Only the difference from the configurationillustrated in FIG. 1 is described below. The image processing deviceillustrated in FIG. 14 has a histogram generation unit 68 additionallyprovided in the configuration illustrated in FIG. 1 described above. Theluminance information generation unit 63 sends the information about theluminance Y also to the histogram generation unit 68.

The histogram generation unit 68 calculates a histogram of the luminancein an image based on the information about the luminance Y from theluminance information generation unit 63. For example, the histogramgeneration unit 68 generates the histogram illustrated in FIG. 8 or FIG.9. The histogram generated by the histogram generation unit 68 is sentto the gamma correction amount calculation unit 67. Accordingly, in thepresent exemplary embodiment, the gamma correction amount calculationunit 67 calculates the correction amount (offset amount) for the gammacharacteristic, based on the histogram and the illuminance informationfrom the illuminance calculation unit 65 described above. The gammacorrection amount calculation unit 67 sets the gamma characteristiccorrected with the correction amount, to the gamma setting unit 66.

Processing executed by the gamma correction amount calculation unit 67according to the present exemplary embodiment is described below indetail. In the present exemplary embodiment, the gamma correction amountcalculation unit 67 calculates an effective range of the luminance fromthe histogram, and determines a luminance range change ratio inaccordance with the illuminance or the gain set in accordance with theilluminance. Then, the gamma correction amount calculation unit 67determines the correction amount for the gamma characteristic based onthe luminance range change ratio.

An example of calculation for the gamma characteristic correction amountby the gamma correction amount calculation unit 67 according to thepresent exemplary embodiment is described with reference to a histogramillustrated in FIG. 15. The histogram illustrated in FIG. 15 is similarto the histogram of the low luminance image illustrated in FIG. 9described above.

The gamma correction amount calculation unit 67 calculates a cumulativefrequency from a lower luminance side in the histogram illustrated inFIG. 15, and sets low effective luminance value Ylow as a value of theluminance Y at which the cumulative frequency Ilow from the lowluminance side reaches a first frequency value. The gamma correctionamount calculation unit 67 sets high effective luminance value Yhigh asa value of the luminance Y at which the cumulative frequency Ihigh fromthe high luminance side reaches a second frequency value, as illustratedin FIG. 15. The first frequency value of the cumulative frequency Ilowfor obtaining the low effective luminance value Ylow and the secondfrequency value of the cumulative frequency Ihigh for obtaining the higheffective luminance value Yhigh may be of different values depending oncharacteristics of the image. For example, when a light source islocated in the field of view, and the image, with low illuminance,includes a local high luminance portion (local bright portion), thelocal high luminance portion may be excluded from the low luminanceimage so that correction tailored for a main object can be achieved. Insuch a case, the cumulative frequency Ihigh of the high luminance areais calculated with the frequency of the local high luminance portioncorresponding to the lighting being excluded.

The gamma correction amount calculation unit 67 determines a luminancerange change ratio Δratio corresponding to the illuminance information(i.e., gain) from a plurality of luminance range change ratios Δratioset in accordance with various gains. As described with reference toFIG. 11, the noise level increases as the gain increases. Accordingly,it is considered that the luminance range can be appropriately correctedwith a higher luminance range change ratio set based on an increase ofthe gain. As described above, the gain and the illuminance informationare highly correlated with each other, so that the luminance rangechange ratio may be obtained in accordance with the illuminanceinformation.

The following Formula (2) represents calculation for the gammaprocessing according to the present exemplary embodiment.YOUT=(255−Ylow−Yhigh)×Δratio/100×γ(YIN)/255+Ylow  Formula (2),where YOUT represents the output luminance value, YIN represents theinput luminance value, and γ( ) represents function corresponding to thereference gamma characteristic 20 illustrated in FIG. 2.

FIG. 16 illustrates the reference gamma characteristic 20 and the offsetgamma characteristic 21 described above in the first exemplaryembodiment, and an example of a corrected gamma characteristic 23according to the second exemplary embodiment. As illustrated in FIG. 16,the gamma correction amount calculation unit 67 sets the low effectiveluminance value Ylow as the offset amount for the low luminance area,and sets the high effective luminance value Yhigh as the offset amountfor the high luminance area. The gamma correction amount calculationunit 67 further performs correction on the luminance range after thecorrection according to the offset amounts Ylow and Yhigh by an amountof Δratio corresponding to the illuminance information. In this manner,the corrected gamma characteristic 23 is calculated. As described above,the image processing unit 6 according to the present exemplaryembodiment executes processing based not only on the illuminanceinformation but also on the effective luminance value of the object, andthus can appropriately correct the luminance range in accordance with acaptured scene for example.

FIG. 17 illustrates a flowchart of processing in the image processingunit 6 according to the present exemplary embodiment. The processing inthe flowchart in FIG. 17 is executed by the illuminance calculation unit65, the gamma correction amount calculation unit 67, and the gammasetting unit 66. The processing illustrated in FIG. 17 may beimplemented by a CPU and the like executing an image processing programaccording to the present exemplary embodiment.

After the camera starts capturing an image, the image processing unit 6starts the processing in the flowchart illustrated in FIG. 17.Processing in steps S1701 and S1702 is similar to that in steps S1001and S1002 in FIG. 10 described above, and description thereof will beomitted. In a case where the gamma correction amount calculation unit 67determines in step S1702 that the illuminance value is equal to or lowerthan the first illuminance value (Yth1) described above (YES in stepS1702), the processing proceeds to step S1704. On the other hand, in acase where the gamma correction amount calculation unit 67 determines instep S1702 that the illuminance value is higher than the firstilluminance value (Yth1) described above (NO in step S1702), theprocessing proceeds to step S1703. In step S1703, the gamma correctionamount calculation unit 67 causes the gamma setting unit 66 to set thereference gamma characteristic 20 illustrated in FIG. 5, as in stepS1003 in FIG. 10 described above. After the processing step S1703 iscompleted, the image processing unit 6 terminates the processing in theflowchart illustrated in FIG. 17.

In step S1704, the gamma correction amount calculation unit 67calculates the cumulative frequency from the low luminance side of thehistogram generated by the histogram generation unit 68 as describedabove, and sets the low effective luminance value Ylow as a value of theluminance Y at which the cumulative frequency Ilow reaches the firstfrequency. After the processing in step S1704 is completed, theprocessing proceeds to step S1705. In step S1705, the gamma correctionamount calculation unit 67 sets the high effective luminance value Yhighas a value of the luminance Y at which the cumulative frequency Ihighcalculated from the high luminance side of the histogram reaches thesecond frequency value. After the processing in step S1705 is completed,the processing proceeds to step S1706.

In step S1706, the gamma correction amount calculation unit 67determines the change ratio Δratio corresponding to the detectedilluminance (i.e., gain) from the plurality of change ratios Δratio ofthe luminance range corresponding to a different one of the gains asdescribed above. FIG. 18 illustrates relationship between the luminancerange change ratios Δratio of and the gains. The gamma correction amountcalculation unit 67 according to the present exemplary embodimentdetermines the luminance range change ratio Δratio corresponding to thedetected illuminance (i.e., gain) from the plurality of luminance rangechange ratios Δratio corresponding to a different one of the gains asillustrated in FIG. 18. Then, the gamma correction amount calculationunit 67 obtains the gamma characteristic 23 as illustrated in FIG. 16for correcting the luminance range, based on the low effective luminancevalue Ylow, the high effective luminance value Yhigh, and the luminancerange change ratio Δratio as described above.

In the example of the present exemplary embodiment, the correction isperformed in such a manner that the luminance range in the highluminance area is compressed by using the luminance range change ratioΔratio. Alternatively, the luminance range may be compressed so that thetotal of the luminance range change ratio in the low luminance area andhigh luminance area becomes the luminance range change ratio Δratio.After the processing in step S1706 is completed, the processing proceedsto step S1707.

In step S1707, the gamma correction amount calculation unit 67 causesthe gamma setting unit 66 to set the gamma characteristic 23 accordingto the present exemplary embodiment. Thus, the gamma setting unit 66corrects the luminance range for the low illuminance image with thegamma characteristic 23. After the processing in step S1707 iscompleted, the processing in the flowchart illustrated in FIG. 17 isterminated.

Other Exemplary Embodiments

The present invention can also be achieved by the process of supplying aprogram for implementing one or more functions of the above exemplaryembodiments to a system or an apparatus via a network or a storagemedium, and causing one or more processors of a computer of the systemor the apparatus to read and execute the program. The present inventionmay be implemented with a circuit (for example, an application specificintegrated circuit (ASIC)) that implements at least one function.

The exemplary embodiments described above are merely an example ofimplementing the present invention, the technical scope of the presentinvention should not be interpreted as being limited by these exemplaryembodiments. Thus, the present invention may be implemented in variousways without departing from the technical concept or the main feature ofthe present invention.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2016-013511, filed Jan. 27, 2016, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image processing device comprising: at leastone memory storing at least one program of instructions; and at leastone processor that executes the at least one program of instructions toimplement: an illuminance acquisition unit configured to acquireilluminance information indicating an illuminance of an image when theimage is captured; and a setting unit configured to set a gradationcharacteristic of a luminance of the image, wherein the setting unitsets the gradation characteristic that has a first output luminancerange in a case where the illuminance information acquired by theilluminance acquisition unit indicates a first illuminance, and sets thegradation characteristic that has a second output luminance range thatis narrower than the first output luminance range in a case where theilluminance information acquired by the illuminance acquisition unitindicates a second illuminance that is lower than a first illuminance.2. The image processing device according to claim 1, wherein the settingunit is configured to obtain, based on the luminance of the capturedimage, at least one of an effective luminance value on a low luminanceside and an effective luminance value on a high luminance side, and toset the gradation characteristic based on the obtained effectiveluminance value and the acquired illuminance information.
 3. The imageprocessing device according to claim 2, wherein the setting unit isconfigured to set a luminance value at which a cumulative frequency fromthe low luminance side of the captured image becomes a first frequencyvalue as the effective luminance value on the low luminance side, andobtains a luminance value at which a cumulative frequency from the highluminance side of the captured image becomes a second frequency value asthe effective luminance value on the high luminance side.
 4. The imageprocessing device according to claim 2, wherein in a case where thecaptured image includes a local high luminance portion, the setting unitis configured to obtain the effective luminance value on the highluminance side with the local high luminance portion being excluded. 5.The image processing device according to claim 2, wherein the settingunit is configured to determine a change ratio corresponding to thedetected illuminance, from among a plurality of change ratios setcorresponding to respective illuminance values, and to correct thegradation characteristic based on the change ratio determined and theeffective luminance value.
 6. The image processing device according toclaim 1, wherein a maximum output luminance value of the second outputluminance range is less than a maximum output luminance value of thefirst output luminance range.
 7. The image processing device accordingto claim 1, wherein the setting unit determines a maximum outputluminance value of the gradation characteristic in accordance with theilluminance information acquired by the illuminance acquisition unit. 8.The image processing device according to claim 1, wherein the settingunit sets the gradation characteristic by correcting a pre-storedreference gradation characteristic in accordance with the illuminanceinformation acquired by the illuminance acquisition unit.
 9. The imageprocessing device according to claim 8, wherein the setting unitdetermines a correction characteristic for correcting the referencegradation characteristic in accordance with the illuminance informationacquired by the illuminance acquisition unit and corrects, based on thecorrection characteristic, the reference gradation characteristic. 10.The image processing device according to claim 9, wherein the settingunit is configured to set the gradation characteristic by offsetting atleast one of the minimum output luminance value and the maximum outputluminance value of the reference gradation characteristic in accordancewith the acquired illuminance information, and, after the offsetting isperformed, interpolating a luminance value between the minimum outputluminance value and the maximum output luminance value.
 11. The imageprocessing device according to claim 10, wherein the offsetting raisesthe minimum output luminance value by a first predetermined amount in acase where the minimum output luminance value is offset, and wherein theoffsetting lowers the maximum output luminance value in a case where themaximum output luminance value is offset.
 12. The image processingdevice according to claim 11, wherein the second predetermined amount islarger than the first predetermined amount.
 13. The image processingdevice according to claim 10, wherein an offset amount for at least oneof the minimum output luminance value and the maximum output luminancevalue is increased as the illuminance decreases.
 14. The imageprocessing device according to claim 10, wherein the setting unit isconfigured to set an offset amount for the minimum output luminancevalue to a value equal to or lower than a predetermined value.
 15. Animage processing method comprising: acquiring illuminance informationindicating an illuminance of an image when the image is captured; andsetting a gradation characteristic of a luminance of the image, whereinthe setting sets the gradation characteristic that has a first outputluminance range in a case where the illuminance information acquired bythe acquiring indicates a first illuminance, and sets the gradationcharacteristic that has a second output luminance range that is narrowerthan the first output luminance range in a case where the illuminanceinformation acquired by the acquiring indicates a second illuminancethat is lower than a first illuminance.
 16. A non-transitory computerstorage medium storing therein a program that when executed by acomputer causes the computer to: acquire illuminance informationindicating an illuminance of an image when the image is captured; andset a gradation characteristic of a luminance of the image, wherein thecomputer sets the gradation characteristic that has a first outputluminance range in a case where the illuminance information acquired bythe acquiring indicates a first illuminance, and sets the gradationcharacteristic that has a second output luminance range that is narrowerthan the first output luminance range in a case where the illuminanceinformation acquired by the acquiring indicates a second illuminancethat is lower than a first illuminance.